Probabilistic and Decision-Theoretic User Modeling in the Context of Software Customization

Research in the field of user modeling has aimed to supersede the current “one-size-fits-all” trend in software development that forces users to change their behaviour according to the preprogrammed functions. This paper discusses aspects of user modeling and its relevance to software customization. In particular, we focus on user modeling techniques that utilize probabilistic and decision-theoretic models. We examine the fundamental limitations of each modeling technique with respect to the goals of user modeling and software customization. Finally, we propose to use partially observable Markov decision processes as the underlying framework to model users for customizing software, and we plan to apply techniques from inverse reinforcement learning and preference elicitation to learn the user’s utility function. We present our prototype of a typing assistant modeled as a partially observable Markov decision process. Lastly, we outline the future directions of how to incorporate inverse reinforcement and preference elicitation work into our project.